Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[IO-1464] add in unknown parsing for WFTypes #653

Merged
merged 1 commit into from
Sep 7, 2023
Merged

[IO-1464] add in unknown parsing for WFTypes #653

merged 1 commit into from
Sep 7, 2023

Conversation

Nathanjp91
Copy link
Contributor

Problem

WFType enum doesn't accept unknown values and this breaks certain testing with older/experimental Workflow types

Solution

Introduce default parsing that accepts unknown values and casting them to an UNKNOWN enum type

Changelog

Workflow stage enum no longer breaks on experimental and testing types

@linear
Copy link

linear bot commented Aug 24, 2023

IO-1464 BUG: Get workflows raises error on meta

During dogfooding, this code:

from darwin.future.meta.client import MetaClient
from darwin.future.meta.objects.dataset import Dataset

client = MetaClient.local()

team = client.team
team_slug = team.slug

datasets = team.datasets.where(name="contains:error-test")
workflow = client.workflows

print(list(workflow))

for dataset in datasets:
    dataset.workflow()

    print(dataset.item_ids(team_slug))

raised this error

ValidationError(
    model='WorkflowListValidator',
    errors=[
        {
            'loc': ('list', 11, 'stages', 3, 'type'),
            'msg': "value is not a valid enumeration member; permitted: 'dataset', 'annotate', 'review', 
'complete', 'discard', 'model', 'webhook', 'archive', 'consensus_test', 'consensus_entrypoint'",
            'type': 'type_error.enum',
            'ctx': {
                'enum_values': [
                    <WFType.DATASET: 'dataset'>,
                    <WFType.ANNOTATE: 'annotate'>,
                    <WFType.REVIEW: 'review'>,
                    <WFType.COMPLETE: 'complete'>,
                    <WFType.DISCARD: 'discard'>,
                    <WFType.MODEL: 'model'>,
                    <WFType.WEBHOOK: 'webhook'>,
                    <WFType.ARCHIVE: 'archive'>,
                    <WFType.CONSENSUS_TEST: 'consensus_test'>,
                    <WFType.CONSENSUS_ENTRYPOINT: 'consensus_entrypoint'>
                ]
            }
        },
        {
            'loc': ('list', 11, 'stages', 4, 'type'),
            'msg': "value is not a valid enumeration member; permitted: 'dataset', 'annotate', 'review', 
'complete', 'discard', 'model', 'webhook', 'archive', 'consensus_test', 'consensus_entrypoint'",
            'type': 'type_error.enum',
            'ctx': {
                'enum_values': [
                    <WFType.DATASET: 'dataset'>,
                    <WFType.ANNOTATE: 'annotate'>,
                    <WFType.REVIEW: 'review'>,
                    <WFType.COMPLETE: 'complete'>,
                    <WFType.DISCARD: 'discard'>,
                    <WFType.MODEL: 'model'>,
                    <WFType.WEBHOOK: 'webhook'>,
                    <WFType.ARCHIVE: 'archive'>,
                    <WFType.CONSENSUS_TEST: 'consensus_test'>,
                    <WFType.CONSENSUS_ENTRYPOINT: 'consensus_entrypoint'>
                ]
            }
        }
    ]
)
Traceback (most recent call last):
  File "/Users/rslota/Projects/v7labs/darwin-py/darwin/future/core/workflows/list_workflows.py", line 30, in list_workflows
    list_of_workflows = WorkflowListValidator(list=response)  # type: ignore
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 2 validation errors for WorkflowListValidator
list -> 11 -> stages -> 3 -> type
  value is not a valid enumeration member; permitted: 'dataset', 'annotate', 'review', 'complete', 'discard', 'model', 'webhook', 'archive', 'consensus_test', 'consensus_entrypoint' (type=type_error.enum; enum_values=[<WFType.DATASET: 'dataset'>, <WFType.ANNOTATE: 'annotate'>, <WFType.REVIEW: 'review'>, <WFType.COMPLETE: 'complete'>, <WFType.DISCARD: 'discard'>, <WFType.MODEL: 'model'>, <WFType.WEBHOOK: 'webhook'>, <WFType.ARCHIVE: 'archive'>, <WFType.CONSENSUS_TEST: 'consensus_test'>, <WFType.CONSENSUS_ENTRYPOINT: 'consensus_entrypoint'>])
list -> 11 -> stages -> 4 -> type
  value is not a valid enumeration member; permitted: 'dataset', 'annotate', 'review', 'complete', 'discard', 'model', 'webhook', 'archive', 'consensus_test', 'consensus_entrypoint' (type=type_error.enum; enum_values=[<WFType.DATASET: 'dataset'>, <WFType.ANNOTATE: 'annotate'>, <WFType.REVIEW: 'review'>, <WFType.COMPLETE: 'complete'>, <WFType.DISCARD: 'discard'>, <WFType.MODEL: 'model'>, <WFType.WEBHOOK: 'webhook'>, <WFType.ARCHIVE: 'archive'>, <WFType.CONSENSUS_TEST: 'consensus_test'>, <WFType.CONSENSUS_ENTRYPOINT: 'consensus_entrypoint'>])

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/rslota/Projects/v7labs/darwin-py/test.py", line 14, in <module>
    print(list(workflow))
  File "/Users/rslota/Projects/v7labs/darwin-py/darwin/future/core/types/query.py", line 159, in __len__
    self.results = list(self.collect())
  File "/Users/rslota/Projects/v7labs/darwin-py/darwin/future/meta/queries/workflow.py", line 28, in collect
    raise DarwinException from exceptions[0]
darwin.exceptions.DarwinException: DarwinException:

Diagnose and fix the issue - likely a missing member of the WFType Enum

owencjones
owencjones approved these changes Aug 24, 2023
View reviewed changes
Copy link
Contributor

@owencjones owencjones left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, all good.

@Nathanjp91 Nathanjp91 merged commit a4bc974 into master Sep 7, 2023
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@owencjones owencjones owencjones approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Nathanjp91 @owencjones